Optical system with dynamic distortion using freeform elements

ABSTRACT

A method for designing an optical system for providing reliable, robust and successful realization of a distortion variation function is presented. In a preferred embodiment, the proposed distortion variation optical system includes at least two non-symmetrical elements, which are moving in the transverse direction. The proposed freeform lens contains two transmissive refractive surfaces. The freeform elements designed with this method have preferably a flat surface and a non-symmetrical freeform surface. The two plano-surfaces are preferably made to face each other, so that a miniature camera can be offered. The value of the non-symmetrical freeform surface is used to produce variable optical power when the two freeform elements undergo a relative movement in the vertical direction. Using this method, an optical system with an active distortion, smaller form factor, and better imaging quality can be obtained.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims the benefit of U.S. Provisional PatentApplication No. 63/059,621, filed on Jul. 31, 2021, entitled “Opticalsystem with dynamic distortion using freeform elements,” the entirecontents of which are incorporated by reference herein.

BACKGROUND OF THE INVENTION

Embodiments of the present invention relate to the field of opticallenses and their design and, more particularly, to an opticalarchitecture of an optical system and associated image processingsoftware having at least one movable freeform optical element that canbe laterally shifted for changing refractive power to realize adistortion variation function.

Optical image capturing systems are found in myriad applications frommedical diagnostic instruments to entertainment, from scientificutilization to civilian use. In optical image capturing systems,spherical lenses or aspheric lenses are the typical components that arecombined with complementary metal oxide semiconductor sensors foroffering high performance image quality.

However, most existing optical image capturing systems commonly do nothave the dynamic optical angular resolution function which is useful inmany different application scenarios. Traditional optical imagecapturing systems using rotationally symmetric components have limitedperformance that cannot meet all current technology development demands.Benefiting from increasing manufacturing capabilities fornon-symmetrical optical surfaces, optical components with freeformsurfaces are now available as an alternative technique for imageapplications.

There has therefore been a long felt need for an optical image capturingsystem suitable for many application scenarios with the function ofdistortion variation in a small form factor.

BRIEF SUMMARY OF THE INVENTION

Embodiments of the present invention present a method to dynamicallycontrol optical distortion of an optical system in a small form factor,which is well suited for applications such as smart phones, securitycameras, automotive cameras, robotic, drones, IoT and other small-scaleimaging systems.

In a preferred embodiment, the use of at least one and often two or morefreeform lenses offers the variation of distortion with a lateralmovement with respect to the optical axis. Such an optical system offersa compact form factor suitable for using in smart mobile devices,security, automotive or other applications. In some embodiments, afreeform component pair is arranged with one behind the other along theoptical axis of the optical system and each freeform element has a flatsurface and a non-symmetrical freeform surface. Specifically, in onepreferred embodiment, the two plano-surfaces are arranged to face eachother to ensure that the freeform pair do not conflict with each other.

In an embodiment, optical distortion variation is continuously providedby the optical system. Movement of one freeform lens relative to theother provides resolution magnification in the central part of the fieldof view, in the mid-zone and at the edge of the field of view.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofa preferred embodiment of the invention, will be better understood whenread in conjunction with the appended drawings. For illustrationpurposes, the drawings show an embodiment which is presently preferred.It should be understood, however, that the invention is not limited tothe precise arrangements and instrumentalities shown.

In the drawings:

FIG. 1 shows an example optical layout of an optical system comprising afreeform element pair with lateral movement perpendicular to an opticalaxis to dynamically change the distortion;

FIGS. 2A and 2B show in perspective view an example of the freeformelement pair as described in FIG. 1 ;

FIG. 3A shows the freeform element pair in a first configuration;

FIG. 3B shows the resulting clear aperture schematic for the freeformelement pair in the first configuration;

FIG. 3C shows a distortion profile created by the freeform element pairin the first configuration, with a maximum resolution in a mid-zonebetween the center and the edge of the field of view;

FIG. 4A shows the freeform element pair in a second configuration;

FIG. 4B shows the resulting clear aperture schematic for the freeformelement pair in the second configuration;

FIG. 4C shows a distortion profile created by the freeform element pairin the second configuration, with a maximum resolution in a zone towardthe edge of the field of view;

FIG. 5A shows the freeform element pair in a third configuration;

FIG. 5B shows the resulting clear aperture schematic for the freeformelement pair in the third configuration;

FIG. 5C shows a distortion profile created by the freeform element pairin the third configuration, with a maximum resolution in a zone towardthe center of the field of view;

FIG. 6 shows an example embodiment in which actuators controlled by acontroller can move the freeform elements; and

FIGS. 7A-7F show multiple examples of configurations of the at least onefreeform element.

DETAILED DESCRIPTION OF THE INVENTION

The words “a” and “an”, as used in the claims and in the correspondingportions of the specification, mean “at least one.”

FIG. 1 shows an example optical layout 100 of an optical system usingtwo movable freeform elements arranged in tandem to dynamically changethe distortion. In this example, the optical system includes awide-angle lens made of six refractive elements all made of plastics,but the optical system according to the present invention is not limitedto any specific number of optical elements, any specific material or anylens shape. For example, in alternative constructions, the opticalsystems could be made of a plurality of optical elements in anycombination of refractive and reflective optical elements and be made ofany combination of plastics, glass, crystal or other types of elements.According to the invention, the plurality of optical elements includesat least one movable freeform element. The optical system could alsohave more or less than six optical elements. The optical systems couldalso include at least one diffractive optical element, an element withmeta-surfaces, a wafer-scale optical element or any other opticalelement that can be used to help form an optical image in one or moreimage planes from the object, including active deformable elements likea deformable mirror, liquid lens, liquid crystal lens or the like.

In this example, the wide-angle half field of view, represented by theangle of the bundle of rays 132 relative to the central bundle of rays130, is 62.5°, representing a full field of view of 125°. However, thisvalue is just an example of a wide-angle field of view, also known as apanoramic field of view, of an optical system designed according to themethod of the present invention. In all embodiments, the optical imagehas a total field of view. In a preferred embodiment, the wide-angletotal field of view is larger than 80°. In other embodiments, thewide-angle field of view is larger than 120°. In other embodiments, thetotal field of view could be of any value, from extremely narrow fieldof view to extremely wide field of view. For example, in some otherembodiments according to the present invention, the total field of viewof the optical system could be well under 50°. In another embodiment,the total field of view could be well over 180°.

In FIG. 1 , the lens has an object side on the left of the layout and animage side on the right of the layout. In this example layout, from theobject side to the image side, the optical system includes a firstelement 110 being rotationally symmetrical, a second element 112 beingrotationally symmetrical, an aperture stop 114, a third element 116being rotationally symmetrical, a fourth element 118 being of freeformshape, a fifth element 120 being of freeform shape, a sixth element 112being rotationally symmetrical, a sensor cover glass 124 that can alsopossibly act as a filter to cut a selected part of the light spectrumand an image plane 126, a plane at which an image sensor isapproximately located. According to the method of the present invention,there can be any number of rotationally symmetrical elements and atleast one freeform optical element. In a preferred embodiment, there areat least two freeform optical elements. The rotationally symmetricalelements can be of any shape, including spherical or aspherical. Theycan be rotationally symmetrical around the optical axis 150 or aroundany other axis. The optical system could also include any number ofcemented elements forming doublets or triplets or the like in order toimprove the performance of the optical system, including image qualityand chromatic aberrations.

In this example, there are two freeform optical elements 118 and 120that can shift laterally perpendicular to the optical axis 150,illustrated in this example by a translation in the Y direction, toadjust the distortion of the system as further described at FIG. 3 toFIG. 5 . In some other embodiments, the number of freeform elementscould be only one or more than two, up to the total number of elements.Also, in addition to the moving freeform elements, there could at leastone freeform optical element that is fixed. In some embodiments, all ofthe optical elements could be of freeform shapes, with some fixed andsome movable. The position of the freeform elements in the layout 100 isalso an example, but these movable freeform elements could be at anyposition, including at the front of the system, before the aperturestop, after the aperture stop or near the image plane. In the examplelayout 100, the two movable freeform optical elements 118, 120 arearranged in a tandem configuration, but in other embodiments, therecould be at least one optical element located between them. In thisexample optical construction, the moving direction of the elements 118and 120 is parallel to the XOZ plane. The freeform elements 118, 120move in opposite orientation in the Y direction in this example, but themovable freeform elements could move in any direction in order to changethe distortion profile of the optical system, including any combinationof translations and rotations. In this example, if the element 118 movesin the positive Y direction, the element 120 moves in the negative Ydirection and inversely.

The lens in the optical layout 100 has a total track length, representedas the distance from the vertex of the first optical element 110 to theimage plane 124. For consumer electronics applications of the lensaccording to the current invention, the total track length is generallyless than 10 mm, but it could be even smaller and be less than 7.5 mm oreven less than 5 mm. The small form factor for this optical system withdynamic distortion profile is possible thanks to the freeform elementsmoving in a transverse direction instead of a longitudinal direction. Bymoving the freeform elements in a transverse direction, like in the Ydirection in this example, instead of the axial Z direction as would bethe case for a classical zoom system, the method according to thepresent invention allows to limit the total track length of the opticalsystem, limiting the total Z-height of corresponding camera modules,which enables use in devices where the height of the camera system islimited, as in a smartphone. In other applications according to thepresent invention, as in an automotive lens or a security lens, thetotal track length could be larger than 10 mm.

The bundle of rays from a central field of view 130 form an image at alocation 140 in the image plane and the bundle of rays from an edgefield of view 132 form an image at a location 142 in the image plane.Not shown on the layout, there are also rays hitting the front surfaceat every other field angle, forming a continuous sampling of the objectbetween the central field of view 130 and the edge field of view 132,these rays forming a continuous image in the image plane 126. The imagein the image plane is not linear with respect to the field angle. Therelation between the image height and the object field angle is calledthe distribution function and is defined in two dimensions because theoptical system comprises at least one non-symmetrical freeform element.By taking the mathematical derivative of the image height as a functionof the object angle along any desired axis, a resolution curve alongthis axis can be obtained. The resolution curve is often calculated inpixels/degree when the image height is given in pixels of the imagesensor and the field angle in degrees or in μm/degree when the imageheight is given in μm and the field angle in degrees. The resolutioncurve shows where the resolution, or the magnification, is minimum andmaximum along a designated axis.

FIG. 2 shows a general indication of the shape of the freeform elements118 and 120 described in FIG. 1 . Again, this representation is only anexample showing freeform surfaces that can be moved to control thedistortion of a lens to change, in real-time, the distribution functionand the resolution curve depending on the required application or theobject scene content, but these freeform elements can be of any shapedepending on the required distribution function. A freeform element isan optical element with at least one of its surfaces being a freeformsurface. In FIG. 2A, the freeform element 200 includes a freeformsurface on the object side and a flat surface on the image side of thecomplete lens while at FIG. 2B, the freeform element 250 includes a flatsurface on the object side and a freeform surface on the image side ofthe complete lens. In this preferred embodiment example, in no waylimiting the scope of the current invention, the two flat surfaces arenext to each other in part to make sure that the freeform pair do notconflict with each other when translated. An air space between the twooptical elements can be of any thickness as required for creating thedesired distribution function. In an alternative embodiment, the twoflat surfaces could be in contact and moving with opposite lateraltranslation. In a further alternative embodiment, the freeform surfacesof the elements 118 and 120 can be oriented facing each other and nestedwith a minimal air gap to form an optical distortion variation.Alternatively, the two freeform surfaces could each face the object sideof the lens or the image side of the lens. Also, there is no absoluterequirement for a flat surface on a freeform element according to themethod of the present invention and the surface on the other side of thefreeform surface on a freeform element could also be a freeform surfaceor any other surface shape.

In this example, since the lenses are designed to be translated alongthe Y direction, the freeform element 200 has an asymmetrical surfaceshape about the XOZ plane, as illustrated by the cuts 210, 212 and 214along the Y direction and a symmetrical shape about the YOZ plane, asillustrated by the cuts 220, 222 and 224 along the X direction. In thisexample, the cuts in the X directions go from convex at 220 to flat at222 and concave at 224, but this is just an example according to themethod of the present invention. Similarly, the freeform element 250 hasan asymmetrical surface shape about the XOZ plane, as illustrated by thecuts 260, 262 and 264 along the Y direction and a symmetrical shapeabout the YOZ plane as illustrated by the cuts 270, 272 and 274 alongthe X direction. In this example, the cuts in the X direction go fromconvex at 274 to flat at 272 and concave at 270, but this is just anexample according to the method of the present invention.

FIG. 3 represents a first example configuration for the two movablefreeform elements from the layout of FIG. 1 , creating a distortionprofile with a maximum resolution in a mid-zone between the center andthe edge of the field of view. At FIG. 3A, for the sake of illustration,the simplified optical layout 300 shows only the two freeform elements310 and 312 from the full layout previously presented at FIG. 1 . Inthis first configuration, there is zero lateral shift between freeformelements 310 and 312 with respect to the optical axis 305. FIG. 3B showsthe resulting clear aperture schematic 320 for these two freeformelements in a central position. In this case, the aperture of bothelements is centered and the resulting clear aperture 330 is centeredand maximum. FIG. 3C then shows the corresponding resolution curve 350for the full optical system when the elements 310 and 312 are in thisfirst configuration. The resolution curve 350 is the mathematicalderivative with respect to the field angle of the distribution functionrelating the image height from the optical axis with respect to thefield of view angle. The resolution curve 350 is normalized in thehorizontal axis with respect to the maximum field of view, resulting ina maximum field of view in this example of 1.0. The resolution curve 350is also normalized in the vertical axis with respect to the minimumresolution, so that the minimum resolution value in this example is 1.0.In this first example configuration, the minimum 360 is located in thecenter. There is also a maximum of resolution 362 which is located in amid-zone between the center of the field of view and the edge of thefield of view. At the edge of the field of view, the resolution 364 islower than at the maximum 362 and higher than at the minimum 360. Thisresulting resolution curve 350 when the freeform elements are in acentered position with respect to the optical axis will be betterunderstood by referring to FIGS. 4 and 5 that show two otherconfigurations for the freeform elements.

FIG. 4 describes a second example configuration using the movablefreeform elements of FIG. 1 , this time for creating a distortionprofile with a maximum resolution toward the edge field of view. At FIG.4A, a simplified optical layout 400 shows only the two freeform elements410 and 412 from the full layout previously presented at FIG. 1 . Inthis second configuration, the freeform element 410 is translated by adistance Y1 in the negative Y direction with respect to the optical axis405 and the freeform element 412 is translated by a distance Y2 in thepositive Y direction with respect to the optical axis. The absolutevalue of the distance Y1 and the distance Y2 could be identical ordifferent depending on the shape of the elements. FIG. 4B shows theresulting clear aperture schematic 420 for these two freeform elementsin the second configuration. In this case, the aperture 430 of element412 is decentered in the positive Y direction and the aperture 432 ofelement 410 is decentered in the negative Y direction. The resultingclear aperture of the system 440 is smaller than the individualapertures 430 or 432. FIG. 4C then shows the corresponding resolutioncurve 450 for the full optical system when the elements 410 and 412 arein this second configuration. The resolution curve 450 is themathematical derivative with respect to the field angle of thedistribution function relating the image height from the optical axiswith respect to the field of view angle. The resolution curve 450 isnormalized in the horizontal axis with respect to the maximum field ofview, resulting in a maximum field of view in this example of 1.0. Theresolution curve 450 is also normalized in the vertical axis withrespect to the minimum resolution, so that the minimum resolution valuein this example is 1.0. In this second example configuration, theminimum 460 is located in the center. There is also a maximum ofresolution 462 which is located closer to the edge of the field of viewcompared to the maximum 362 from FIG. 3C. The shift of the resolutioncurve is the result of the light rays hitting the freeform surfaces atdifferent locations compared to the first configuration. At the edge ofthe field of view, the resolution 464 is lower than at the maximum 462and higher than at the minimum 460, but in other embodiments, themaximum resolution value could be at the very edge of the field of view.

FIG. 5 describes a third example configuration using the movablefreeform elements of FIG. 1 , this time for creating a distortionprofile with a maximum resolution in the center of the field of view. AtFIG. 5A, a simplified optical layout 500 shows only the two freeformelements 510 and 512 from the full layout previously presented at FIG. 1. In this third configuration, the freeform element 510 is translated bya distance Y3 in the positive Y direction with respect to the opticalaxis 505 and the freeform element 512 is translated by a distance Y4 inthe negative Y direction with respect to the optical axis. The absolutevalue of the distance Y3 and the distance Y4 could be identical ordifferent depending on the shape of the elements. FIG. 5B shows theresulting clear aperture schematic 520 for these two freeform elementsin the third configuration. In this case, the aperture 530 of element510 is decentered in the positive Y direction and the aperture 532 ofelement 512 is decentered in the negative Y direction. The resultingclear aperture of the system 540 is smaller than the individualapertures 530 or 532. FIG. 5C then shows the corresponding resolutioncurve 550 for the full optical system when the elements 510 and 512 arein this third configuration. The resolution curve 550 is themathematical derivative with respect to the field angle of thedistribution function relating the image height from the optical axiswith respect to the field of view angle. The resolution curve 550 isnormalized in the horizontal axis with respect to the maximum field ofview, resulting in a maximum field of view in this example of 1.0. Theresolution curve 550 is also normalized in the vertical axis withrespect to the minimum resolution, so that the minimum resolution valuein this example is 1.0. In this third example configuration, the minimum564 is located in the edge of the field of view and the maximum ofresolution 560 is located in the center of the field of view. The shiftof the resolution curve compared to FIG. 3 and FIG. 4 is the result ofthe light rays hitting the freeform surfaces at different locationscompared to the first or second configurations.

The three example configurations from FIG. 3 to FIG. 5 showed threediscrete positions of the freeform elements creating three differentresolution curves. In a preferred embodiment, the translation of thefreeform elements is continuous and there is a resulting resolutioncurve for each of the continuous positions, allowing to create aresolution curve with a continuously movable maximum resolution valueacross the field of view. Alternatively, in some other embodiments, thefreeform positions creating an image could be limited to only somediscrete positions so that only a pre-determined number ofpre-calculated resolution curves could be achieved. In a preferredembodiment, the resolution value, expressed for example inmicrometers/degree when the image height is expressed in micrometers andthe field angle is expressed in degrees, calculated for at least onefield angle in the total field of view, varies by at least ±10% betweena first resolution curve and a second resolution curve. The firstresolution curve could be the one with the minimum resolution value,when the freeform elements are in a first configuration. The secondresolution curve could be the one with the maximum resolution value whenthe freeform elements are in a second configuration. For example, if anoptical system with the freeform elements positioned in a firstconfiguration has a resolution of 30 μm/° at an object field angle forexample of 15° from the optical axis and the same optical system withthe freeform elements positioned in a second configuration has aresolution of 45 μm/° at the same object field angle of 15°, there is anincrease of resolution of +50% using the formula(MaxValue−MinValue)/MinValue. This value of +50% is well above theminimum requirement of ±10% according to the present invention. In otherembodiments according to the present invention, the resolution varies byat least ±25% or even at least ±50% between its minimum resolution valuewhen the freeform elements are in a first configuration compared to itsmaximum resolution value when the freeform elements are in a secondconfiguration.

FIG. 6 shows an example embodiment of the optical system to createdistorted images having dynamically adapting non-linear resolutioncurves in which actuators controlled by a controller can move thefreeform elements. The full optical and electronic system 600 includes amechanical casing 610 holding the optical elements shown previously atFIG. 1 together. This mechanical casing can be of any material,including metal or plastics. In all embodiments, there is at least oneactuator configured to move the at least one movable freeform elementbetween at least a first position and a second position different fromthe first position. In the figure, there is an actuator 630 that canmove the freeform element 620 in the positive or negative Y directionand an actuator 635 that can move the freeform element 625 in thepositive or negative Y direction. The actuators moving the freeformelements can be of any shape or using any technology, includingmicro-electromechanical systems, voice coil motors, shape memory alloys,linear actuators, rotary actuators, hydraulic actuators, pneumaticactuators, electric actuators, thermal actuators, magnetic actuators,mechanical actuators, or the like. In some embodiments, these actuatorscan be completely enclosed inside the mechanical casing 610 while inother embodiments, a part of the actuators could extend outside of themechanical casing 610 via a hole in the Y direction. In some otherembodiments, the two freeform elements could be moved together inopposite directions with a single actuator and a mechanism linking theat least 2 freeform elements together. In all embodiments, there is alsoa controller configured to control the at least one actuator. In thefigure, the actuators 630 and 635 are linked to a controller 650 vialinks 640 and 645. The controller could be any hardware device capableof controlling the position of the freeform elements depending on theresolution curve required, including depending on the particularapplication of the lens, movement detected in the object, particularobjects of interest identified in the image, information from internalor external sensors like a compass, a GPS or an accelerometer, a humanuser selection (either by conscious direct input or unconscious input aseye-gazing selection), an algorithm automatically analyzing theresulting image and automatically selecting the best resolution curve,or any other way to decide of the best resolution curve at a given time.The controller can be located inside the lens system, inside the camera,or inside the device in which the camera is located, like a smartphoneor a computer. Alternatively, the controller could also be part of aseparate device and the information about the ideal actuator positioncan be transmitted via any communication link, including wireless, viathe Internet or the like.

When the controller instructs the actuator to move the freeform element,it creates a resolution curve as desired. According to the presentinvention, when the at least one movable freeform element is in thefirst position, the optical image created by the plurality of opticalelements has a first resolution curve and when the at least one movablefreeform element is in the second position, the optical image created bythe plurality of optical elements has a second resolution curve, thefirst resolution curve being different than the second resolution curve.In the system, there is a plurality of optical elements configured tocreate an optical image at one or more image planes in which the one ormore image sensors 615 are located. The case when there is more than oneimage plane and more than one imaging sensor could be for example, butnot limited to, a system in which the red, the green and the blue colorsare separated in the system to be imaged on multiple sensors. Theplurality of optical elements includes at least one movable freeformelement. In some embodiments, the optical system includes an imagesensor located at the image plane and the image sensor is configured toconvert the optical image created by the plurality of optical elementsinto a digital image. The image sensor transforms the optical image to adigital image by reading and converting the pixels from the imagesensor. Here, the image sensor can be of any kind, including a CMOS orCCD creating a monochromatic or polychromatic image, but can also be anyother kind of sensor creating some sort of data from focused rays oflight in an image plane of the optical system like a lidar using asingle-photon avalanche diode, avalanche photodiode array or any othertype of lidar sensor, a time of flight or any other type of depthsensor, a temperature sensor or any other kind of electronic sensorconverting the input light to a digital image, either inside a digitalimage file recorded with any file format (JPG, BMP, TIF, PNG, GIF or anyother digital image file format) or a digital image inside a memorybuffer. The input light can be of any kind and from any part of thespectrum, including, but not limited to, visible light, ultravioletlight, infrared light, polarized light, light from a laser source, lightfrom diffractive elements or the likes. With any of these applications,the term distorted optical image created by the optical system must beunderstood in this patent to comprise any kind of optical image formedby the convergence of ray of light in an image plane, including, but inno way limited to, a monochromatic or polychromatic images, temperatureimage, depth images, multispectral information images. Same for thecorresponding digital image file from the image sensor, it must beunderstood in this patent to comprise any kind of digital image fileresulting from the above-mentioned types of optical image or types ofimage sensors and is not restricted to any kind of image.

The digital image file or memory buffer is linked to a processor 670 viaa link 660. The processor 670 is any hardware able to analyze andprocess a digital image file, including a central processing unit (CPU),a graphical processing unit (GPU), an application-specific integratedcircuit (ASIC), a field programmable gate array (FPGA) or any otherhardware capable to process the digital image file. The processor 670can be located inside the lens system, inside the camera, or inside thedevice in which the camera is located, like a smartphone or a computer.Alternatively, the processor could also be part of a separate device andthe information for the controller can be transmitted via anycommunication link, including wireless, via the Internet or the like. Insome embodiments according to the present invention, the processor isconfigured to process the digital image. When doing so, this processor670 is configured to analyze and process the digital image from theoptical system and can interpret automatically how the resolution curveshould be modified in order to improve the digital image file receivedfrom the image sensor 615. The processor can then instruct thecontroller to cause the at least one actuator to move the at least onemovable freeform element to a different position by transmitting theinformation to the controller 650 via a link 680. The link 680 is anyway to transfer information from a device to another, including aphysical cable, over the air, via the Internet or the like. Thecontroller then updates the position of the freeform elements 620 and625 with the actuators 630 and 635 so that the optical system is in anew configuration. After the position of the freeform elements isadjusted, the next image captured by the image sensor 615 has the newrequired resolution curve requested by the processor 670. Theseiterative steps can continue in real-time to change continuously theresolution curve depending on the application and the analyzed contentof the digital image file, either in a closed-loop or an open-loopoptimization process.

In some embodiments according to the present invention, the processor670 can also process the received digital image file in order to eitherdisplay it to a human observer or to send it to a further algorithm unitthat is using the output image such as, but not limited to, any computervision algorithm, any artificial intelligence algorithm, any machinelearning algorithm, any depth calculation algorithm (including fromstereo image, structured light analysis or from time of flight sensors),any further image processing or image enhancement algorithm or any otheralgorithm that can use the output from the optical system. This furtherstep of processing of the digital image could include at least partialimage dewarping in order to at least partially correct the non-lineardistortion in the image resulting from the adaptable resolution curve.Here, dewarping is understood to mean any correction, modification,processing or transformation of the optical distortion in a digitalimage. This image dewarping step is possible when the processor 670knows the distribution function or the resolution curve of the opticalsystem at the time the digital image file is captured. With calibrationor with a lookup table, the processor 670 can know the exact distortionprofile for each position of the freeform surfaces and can optimallyprocess the images. In addition to processing the image distortion, thisprocessing step could also include any other kind of image processingalgorithm, including correction of other aberrations in the opticalsystems like the ones that vary dynamically with the position of thefreeform surfaces. When the processor 670 instructs the controller 650of a given configuration, the information about the distributionfunction or the resolution curve can be selected from a lookup table, apolynomial function with adjusted coefficients or any other way to storethe information. Alternatively, when the controller 650 is configuredfrom an external source other than the processor 670 to place thefreeform elements in a given position to create a specific distributionfunction and resolution curve, the controller 650 can send theinformation about the distribution function or the information about thefreeform position to the processor. Alternatively, the camera of theoptical system could also write this information in a marker or awatermark visible or not inside the digital image file or in a meta-dataof the digital image file.

The example optical system presented at FIG. 6 can also be used in amore general method to create distorted images having dynamicallyadapting non-linear resolution curves. The method includes the steps offirst creating an optical image in an image plane of the plurality ofoptical elements, the optical image having a total field of view andthen causing, by the controller, the at least one actuator to move theat least one movable freeform element. With this method, when the atleast one movable freeform element is in a first position, the opticalimage has a first resolution curve and when the at least one movablefreeform element is in a second position, the optical image has a secondresolution curve. In this method, the first resolution curve isdifferent than the second resolution curve.

FIG. 7 shows multiple examples of configurations of the at least onefreeform element. These are just some examples to better understand thepossibilities, in no way limiting the scope of the present invention tothe shown examples. At FIG. 7A, the configuration with two freeformelements 700 and 705, each having a plano surface and a freeformsurface, the two plano surfaces facing each other, is shown. At FIG. 7B,the configuration with a single freeform element 710 having one freeformsurface and one plano surface is shown. At FIG. 7C, the configurationwith a single freeform element 720 having two freeform surfaces isshown. At FIG. 7D, the configuration with two freeform elements 730 and735, each having a freeform surface and a non-freeform surface is shown.The non-freeform surface could be of any shape, including spherical,aspherical or the like. At FIG. 7E, the configuration with two freeformelements 740 and 745, each having a plano surface and a freeformsurface, the two freeform surfaces facing each other, is shown. At FIG.7F, the configuration with two freeform elements 750 and 760, eachhaving a plano surface and a freeform surface, with at least one otheroptical element located between the movable freeform elementsrepresented by the element 755, is shown. This at least one otheroptical element located between the movable freeform elements could beof any shape, including plano, spherical, aspherical, cylindrical,freeform or any other shape.

In some other embodiments of the present invention, the translation ofthe freeform elements in the Y direction to control the resolution curveof the optical system could be combined with translation in the X or Zdirections or rotation around any axis of at least one of the freeformoptical elements for better control on the lens performance (6 degreesof freedom), including keeping or changing the lens total field of viewor improving the image quality by increasing the resulting imagemodulation transfer function (MTF). In some other embodiments, therecould also be at least one more movable optical surface in the opticalsystem, whether a freeform element or an element with rotationalsymmetry, that could be moved in either the X, Y or Z direction in orderto better control the image quality, resolution curve or any otheroptical performance of the system. This can also be combined with atranslation of the image sensor, including an auto-focus system, inorder to maximize the image quality in any region desired for a givenconfiguration of the freeform elements.

In some embodiments of the present invention, the aperture stop couldinclude an adjustable IRIS that can be controlled electronically. Bychanging the size of the aperture depending on the configuration of thefreeform optical elements, a better control of the aberrations ispossible. For example, in no way limiting the scope of the presentinvention, the aperture stop diameter could be smaller when the freeformelements are in an off-centered position relative to the optical axisand larger when the freeform elements are centered around the opticalaxis.

In all of the embodiments according to the method of the presentinvention, the resolution curves in either the X or the Y direction canbe symmetrical or not around the value at the center of the field ofview, depending on the shape of the freeform elements.

In all of the embodiments according to the method of the presentinvention, the movable freeform surfaces can have an optical coatingapplied to them. This coating can vary across the optical surface,allowing to change the optical properties of the systems when moving atleast one freeform surface since the part of the surface across thelight path has a variable optical coating offering variableperformances.

In other embodiments of the present invention, the optical system is aprojection system projecting an image outside of the optical systeminstead of an imaging system creating an optical image on an imagesensor of the optical system. In that case, an alternate distributionfunction and an alternate resolution curve can be defined, this timelinking the angular position of the image to the height of the sourcepoint on the projecting object. In that projection case instead ofimaging case, the method to adjust the resolution curve by movingfreeform optical elements according to the present invention remains thesame. In this kind of system, the object is the display source forcreating the projection and the optical image is created by theprojected rays in the scene around the optical system. In thisprojection case, the image plane might be different for each fieldangle. For example, each object in the scene around the optical systemreceiving a ray of light from the projection system may define aseparate image plane. In such a case, the optical image may bedistributed across the optical image planes in the scene. This way, theprojection system can adjust the distortion of the system to match thedisplay shape in real-time even if the display shape change in time oraccommodate the size and resolution of the displayed object depending onthe application scenario. For example, the system according to thepresent invention could be used in a projection system for an automotiveapplication in which the projection system with dynamic distortion couldbe used for example to project signs on the road of different size orlight intensity that vary according to the application. It could also beused to project a light source in a time-of-flight system or astructured light system. By changing the projection distortiondynamically, the system could be used to increase the amount of light orresolution of the target in a specific zone of interest, allowing toimprove the calculation of distance to a target in front of theautomotive vehicle based on the detected targets. The system could runin an open-loop or closed-loop to improve the performance on a zone ofinterest based or not on feedback from an algorithm unit.

In other embodiments according to the present invention, instead of onlytranslation of the freeform elements, the actuators could move thefreeform elements in a purely rotational movement around any axis,including tilt movement of the freeform elements. More generally, theactuators can move the freeform elements in any combination of rotationaround any axis, any tilt and any translation in any direction.According to the present invention, the freeform elements can move with6 degrees of freedom in the optical system as needed.

In other embodiments according to the present invention, the opticalsystem is used to capture at least a first image with higher resolutionin a first region of interest and a second image with higher resolutionin a second region on interest, the second region of interest beingdifferent than the first region of interest. The at least two imageswith higher resolution in their respective zone of interest are thencombined into a single image by a hardware or software image processingalgorithm to create a combined image having higher resolution in the atleast two zones of interest by using the image with the highestresolution in each zone of the combined image.

In other embodiments according to the present invention, the opticalsystem with dynamic distortion is running in a loop with the processoranalyzing the resulting digital image. In that case, it is the processorthat instructs the controller of a new position for the movable freeformelement after processing of the digital image. For example, the opticalsystem could be used in a device that automatically optimizes thedistortion profile of the optics in order to maximize the ability torecognize an object in the scene or any other kind of information thealgorithm requires. The algorithm could work on maximizing anyparameter, for example the confidence level of a deep learningartificial intelligence algorithm recognizing an object in the scene.The processor would then instruct the controller to move the movablesfreeform elements until the best configuration is achieved for thedesired output, similar to what an autofocus trying to maximize thefocus of a camera does but this time optimizing on the distortioninstead of the focus position.

In a preferred embodiment of the current invention, the total field ofview of the optical image is kept the same when the resolution curve ischanged. More precisely, for at least two positions of the at least onemovable freeform element, the total field of view of the optical imagewhen the at least one movable freeform element is in the first positionis the same as the total field of view of the optical image when the atleast one movable freeform element is in the second position. In otherembodiments, in addition to changing the resolution curve, the totalfield of view of the optical image could be changed by the move of atleast one freeform optical element.

There are multiple examples of applications for such an optical systemwith dynamic optical distortion. In the consumer electronics industry,including for mobile phones, tablet, computers or other smallelectronics, the use of dynamic distortion allows to replace with asingle optical system what would require either multiple cameras toimage each zone of interest of the field of view with equal resolutionor either a larger image sensor which increases the size and cost of thesystems and is not desirable especially in consumer electronicsapplications. By using freeform surfaces that move in the transversedirection instead of the longitudinal direction, this allows reducingthe size of the system even more, allowing to fit even in the thinnestof the consumer electronics devices like thin mobile phones. In securityor automotive applications, having a dynamic distortion allowing tocreate a zone of interest with increased number of pixels in real-timeis useful to better inspect objects seen in that zone, whether it is anintruder seen by a security camera or an obstacle on the road ahead inan automotive camera. Again, by using a single optical system withdynamic distortion, the system according to the present invention allowsto either replace multiple cameras with a single camera or to use animage sensor with a lower total number of pixels. This allows either toreduce the size and cost by using smaller overall systems when a lowernumber of total pixels is required or to increase the low-lightsensitivity of the optical systems by using larger pixels on the imagesensor since again a less overall number of pixels is required. Otherapplications for an optical system designed according to the method ofthe present invention includes industrial applications, aerospace,medical or any other application that can benefit from a system withdynamic optical distortion in a compact form factor. One other exampleof application of an optical system with dynamic distortion could be inmachine perception and robotics, for example in a system to mimicdifferent types of bioinspired vision systems. One example is a humaneye that is able to adapt the vision to see larger objects when needed,farther objects when needed or to direct the zone of higher resolutionto the direction of the gaze. Such types of optical systems with dynamicdistortion using freeform elements can be used to replicate for examplehuman vision systems for certain applications such as AR/VR/MR headsets.In other machine perception application, other bio inspired visionsystems can be designed to expand the machine perception capabilitiesbeyond human vision and perception.

All of the above figures and examples show the method to adjust thedistribution function and its related resolution curve by using at leastone movable freeform element and at least two freeform elements in apreferred embodiment. These examples are not intended to be anexhaustive list or to limit the scope and spirit of the presentinvention. It will be appreciated by those skilled in the art thatchanges could be made to the embodiments described above withoutdeparting from the broad inventive concept thereof. It is understood,therefore, that this invention is not limited to the particularembodiments disclosed, but it is intended to cover modifications withinthe spirit and scope of the present invention as defined by the appendedclaims.

We claim:
 1. An optical system to create distorted images havingdynamically adapting non-linear resolution curves, the optical systemcomprising: a. a plurality of optical elements configured to create anoptical image at one or more image planes, the optical image having atotal field of view, the plurality of optical elements including atleast one movable freeform element; b. at least one actuator configuredto move the at least one movable freeform element between at least afirst position and a second position different from the first position;c. a controller configured to control the at least one actuator, whereinwhen the at least one movable freeform element is in the first position,the optical image created by the plurality of optical elements has afirst resolution curve and when the at least one movable freeformelement is in the second position, the optical image created by theplurality of optical elements has a second resolution curve, the firstresolution curve being different than the second resolution curve. 2.The optical system of claim 1, further comprising an image sensorlocated at the image plane, the image sensor being configured to convertthe optical image created by the plurality of optical elements into adigital image.
 3. The optical system of claim 2, further comprising aprocessor configured to process the digital image.
 4. The optical systemof claim 3, wherein the processing of the digital image includes atleast partially dewarping the digital image.
 5. The optical system ofclaim 3, wherein the processor is configured to, after processing of thedigital image, instruct the controller to cause the at least oneactuator to move the at least one movable freeform element to adifferent position.
 6. The optical system of claim 1, comprising twomovable freeform elements.
 7. The optical system of claim 1, wherein thetotal field of view is greater than 80°.
 8. The optical system of claim1, wherein the total field of view of the optical image when the atleast one movable freeform element is in the first position is the sameas the total field of view of the optical image when the at least onemovable freeform element is in the second position.
 9. The opticalsystem of claim 1 having a total track length, the total track lengthbeing less than 10 mm.
 10. The optical system of claim 1, wherein aresolution value for at least one field angle in the total field of viewvaries by at least ±10% between the first resolution curve and thesecond resolution curve.
 11. The optical system of claim 1, wherein theoptical image created by the plurality of optical elements is at leastone of a monochromatic image, a polychromatic image, a temperatureimage, a depth image, or a multispectral information image.
 12. Theoptical system of claim 1, wherein the plurality of optical elements areconfigured to accept input light in a visible range.
 13. A method tocreate, by an optical system, distorted images having dynamicallyadapting non-linear resolution curves, the optical system comprising aplurality of optical elements having a total field of view and includingat least one movable freeform element, at least one actuator for movingthe at least one movable freeform element, and a controller, the methodcomprising: a. creating, by the plurality of optical elements, anoptical image in one or more image planes, the optical image having atotal field of view; and b. causing, by the controller, the at least oneactuator to move the at least one movable freeform element, wherein whenthe at least one movable freeform element is in a first position, theoptical image has a first resolution curve and when the at least onemovable freeform element is in a second position, the optical image hasa second resolution curve, the first resolution curve being differentthan the second resolution curve.
 14. The method of claim 13, whereinthe optical system further comprises an image sensor located at theimage plane, the method further comprising converting, by the imagesensor, the optical image into a digital image.
 15. The method of claim14, wherein the optical system further comprises a processor, the methodfurther comprising processing, by the processor, the digital image. 16.The method of claim 15, wherein the processing of the digital imageincludes at least partially dewarping the digital image.
 17. The methodof claim 15, further comprising instructing, by the processor afterprocessing the digital image, the controller to cause the at least oneactuator to move the at least one movable freeform element to adifferent position.
 18. The method of claim 13, wherein the opticalsystem comprises two movable freeform elements.
 19. The method of claim13, wherein the total field of view is greater than 80°.
 20. The methodof claim 13, wherein the total field of view of the optical image whenthe at least one movable freeform element is in the first position isthe same as the total field of view of the optical image when the atleast one movable freeform element is in the second position.
 21. Themethod of claim 13, wherein the optical system has a total track length,the total track length being less than 10 mm.
 22. The method of claim13, wherein a resolution value for at least one field angle in the totalfield of view varies by at least ±10% between the first resolution curveand the second resolution curve.
 23. The method of claim 13, wherein theoptical image created by the plurality of optical elements is at leastone of a monochromatic image, a polychromatic image, a temperatureimage, a depth image, or a multispectral information image.
 24. Themethod of claim 13, wherein the plurality of optical elements areconfigured to accept input light in a visible range.